Skip to content

Comments

cannon: Migrate fuzz tests#16851

Merged
mbaxter merged 15 commits intodevelopfrom
mbax/issue-16483/fuzz-tests
Aug 11, 2025
Merged

cannon: Migrate fuzz tests#16851
mbaxter merged 15 commits intodevelopfrom
mbax/issue-16483/fuzz-tests

Conversation

@mbaxter
Copy link
Contributor

@mbaxter mbaxter commented Jul 28, 2025

Description

Migrate fuzz tests to use the DiffTester framework introduced here.

As part of this migration, tweaked the DiffTester to improve performance:

  • Allow VersionedVMTestCase objects to be injected once to save time on setup (these are expensive to generate because we are loading / reading contract data)
  • Cache test setup objects that are created in generateTestModifiers()

And made a few other adjustments to support the tests:

  • Allow the randomSeed to be injected into DiffTester
  • Give the PostStateCheckFn access to the final StepWitness produced

Metadata

Part of #16483

@codecov
Copy link

codecov bot commented Jul 28, 2025

Codecov Report

❌ Patch coverage is 94.66667% with 4 lines in your changes missing coverage. Please review.
✅ Project coverage is 66.97%. Comparing base (df12f4d) to head (c18ed48).
⚠️ Report is 13 commits behind head on develop.

Files with missing lines Patch % Lines
cannon/mipsevm/tests/difftester.go 94.66% 3 Missing and 1 partial ⚠️

❗ There is a different number of reports uploaded between BASE (df12f4d) and HEAD (c18ed48). Click for more details.

HEAD has 4 uploads less than BASE
Flag BASE (df12f4d) HEAD (c18ed48)
cannon-go-tests-64 3 1
contracts-bedrock-tests 2 0
Additional details and impacted files
@@             Coverage Diff              @@
##           develop   #16851       +/-   ##
============================================
- Coverage    81.85%   66.97%   -14.88%     
============================================
  Files          161       55      -106     
  Lines         9219     4660     -4559     
============================================
- Hits          7546     3121     -4425     
+ Misses        1529     1394      -135     
- Partials       144      145        +1     
Flag Coverage Δ
cannon-go-tests-64 66.97% <94.66%> (-0.54%) ⬇️
contracts-bedrock-tests ?

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines Coverage Δ
cannon/mipsevm/tests/difftester.go 90.67% <94.66%> (+0.42%) ⬆️

... and 113 files with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@mbaxter mbaxter force-pushed the mbax/issue-16483/singleton-tests branch from ccaf5b5 to 3038d38 Compare July 29, 2025 15:45
@mbaxter mbaxter force-pushed the mbax/issue-16483/fuzz-tests branch from a844164 to 3ae520a Compare July 29, 2025 15:48
@mbaxter mbaxter force-pushed the mbax/issue-16483/singleton-tests branch from 3038d38 to cc788c1 Compare July 30, 2025 19:05
@mbaxter mbaxter force-pushed the mbax/issue-16483/fuzz-tests branch 2 times, most recently from ceb5646 to 9a6ba44 Compare July 30, 2025 22:14
@mbaxter mbaxter force-pushed the mbax/issue-16483/singleton-tests branch from cc9efeb to 023c274 Compare August 1, 2025 19:44
Base automatically changed from mbax/issue-16483/singleton-tests to develop August 1, 2025 20:20
@mbaxter mbaxter force-pushed the mbax/issue-16483/fuzz-tests branch from 90468eb to c18ed48 Compare August 1, 2025 20:23
@mbaxter mbaxter requested a review from Inphi August 1, 2025 20:47
@mbaxter mbaxter marked this pull request as ready for review August 1, 2025 20:47
@mbaxter mbaxter requested review from a team as code owners August 1, 2025 20:47
@mbaxter mbaxter added this pull request to the merge queue Aug 11, 2025
Merged via the queue into develop with commit ba27c30 Aug 11, 2025
65 checks passed
@mbaxter mbaxter deleted the mbax/issue-16483/fuzz-tests branch August 11, 2025 16:06
theochap pushed a commit to theochap/optimism that referenced this pull request Aug 19, 2025
* Migrate first fuzz test

Add vm version caching, randomSeed configuration to DiffTester

* Move diffTester setup outside of fuzz function

* Simplify FuzzStateSyscallBrk

* Migrate FuzzStateSyscallMmap

* Standardize fuzz TestOption's

* Migrate FuzzStateSyscallExitGroup

* Migrate FuzzStateSyscallFcntl

* Migrate FuzzStateHintRead, pass step witness to PostCheckFn

* Migrate FuzzStatePreimageRead

* Migrate FuzzStateHintWrite

* Migrate FuzzStateConsistencyMulOp

* Migrate multiplication fuzz tests

* Migrate FuzzStatePreimageWrite

* Reuse / cache test setup when generating modifications

* Fix FuzzStateHintWrite perf - calculate hint data inside of tests
leopoldjoy pushed a commit to leopoldjoy/optimism that referenced this pull request Aug 22, 2025
* Migrate first fuzz test

Add vm version caching, randomSeed configuration to DiffTester

* Move diffTester setup outside of fuzz function

* Simplify FuzzStateSyscallBrk

* Migrate FuzzStateSyscallMmap

* Standardize fuzz TestOption's

* Migrate FuzzStateSyscallExitGroup

* Migrate FuzzStateSyscallFcntl

* Migrate FuzzStateHintRead, pass step witness to PostCheckFn

* Migrate FuzzStatePreimageRead

* Migrate FuzzStateHintWrite

* Migrate FuzzStateConsistencyMulOp

* Migrate multiplication fuzz tests

* Migrate FuzzStatePreimageWrite

* Reuse / cache test setup when generating modifications

* Fix FuzzStateHintWrite perf - calculate hint data inside of tests
theochap pushed a commit to theochap/optimism that referenced this pull request Aug 26, 2025
* Migrate first fuzz test

Add vm version caching, randomSeed configuration to DiffTester

* Move diffTester setup outside of fuzz function

* Simplify FuzzStateSyscallBrk

* Migrate FuzzStateSyscallMmap

* Standardize fuzz TestOption's

* Migrate FuzzStateSyscallExitGroup

* Migrate FuzzStateSyscallFcntl

* Migrate FuzzStateHintRead, pass step witness to PostCheckFn

* Migrate FuzzStatePreimageRead

* Migrate FuzzStateHintWrite

* Migrate FuzzStateConsistencyMulOp

* Migrate multiplication fuzz tests

* Migrate FuzzStatePreimageWrite

* Reuse / cache test setup when generating modifications

* Fix FuzzStateHintWrite perf - calculate hint data inside of tests
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants